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1. Introduction 


The Australian Bureau of Statistics (ABS) is Australia's official national 
statistical agency. The ABS' mission is to assist and encourage informed decision 
making, research and discussion within governments and the community, by 
leading a high quality, objective and response national statistical service. 
However, it is recognised that "Confidence in the quality of the information it 
produces is a survival issue for a statistical agency" Brackstone (1999) and the 
ABS maintains this trust with our users through the use of frameworks for 
ensuring the quality of our statistics, amongst other quality assurance initiatives. 


One of these frameworks is the Australian Bureau of Statistics Data Quality 
Framework. The ABS Data Quality Framework comprises of seven dimensions 
based on Statistics Canada's Quality Assurance Framework and the European 
Statistics Code of Practice. The seven dimensions are Institutional Environment, 
Relevance, Timeliness, Accuracy, Coherence, Interpretability and Accessibility. 
The dimension of Institutional Environment comes from the European Statistics 
Code of Practice and was added to the ABS Data Quality Framework in 
December 2007 after much discussion and debate amongst senior managers. It 
had long been acknowledged by senior managers that trust and credibility of an 
organisation were aspects of quality that should be considered for assessment 
purposes. The increasing use of administrative data by the ABS contributed to 
the inclusion of Institutional Environment to the Data Quality Framework. It was 
felt that this dimension needed to be included to ensure greater transparency of 
the quality of the data being used in ABS output given the increasing use of data 
that were not strictly from ABS collections. 


' The views expressed are attributable to the authors and do not necessarily reflect those of the 
Australian Bureau of Statistics. 


Dennis Trewin (2001) had already foreshadowed the development of quality 
declarations in 2001 when he wrote about the presentation of statistics on the 
internet which included the need to educate the user community on the quality 
of official statistics, "There is an obligation on us [ABS] to provide a range of 
supporting information on data outputs, including that on quality, so that they 
[users] can make their own judgements on fitness of use". The ABS externalised 
the Data Quality Framework on the ABS website initially through reporting the 
quality of their statistical products in the form of Quality Declarations from 
January 2008. Quality declarations provide a synopsis of the quality of the 
statistics being accessed according to the seven dimensions of the ABS Data 
Quality Framework. Quality declarations were designed for electronic web 
release only. They were a part of the ABS' electronic data and metadata vision 
project which included the move away from paper publications to web based 
releases only. Quality declarations use hyperlinks to more detailed information 
where applicable and are themselves accessible through a link near the heading 
of the statistical release on the ABS website. They were designed to complement 
but not replace existing metadata sources on the web (e.g. Explanatory notes, 
Concepts sources and methods, Technical notes), which are more detailed. The 
detail that is provided within each dimension of the quality declaration may differ 
across quality declarations because all collections are different and hence have 
different quality aspects that need to be declared. For example surveys of carers 
or indigenous populations are likely to have coverage issues and surveys with 
sensitive topics may have response bias. 


A more definitive explanation of the ABS Data Quality Framework and its 
uses was released in May 2009 to the ABS website. The paper, ABS Data Quality 
Framework, May 2009 (cat no. 1520.0), described the dimensions of the ABS 
Data Quality Framework in detail to enable its use in defining the quality of a 
data item or collection of data items; assessing data in the context of a data 
need; and identifying data gaps and areas of future improvement. 


2. External use of the ABS Data Quality Framework for Assessment 
and the ABS Role 


Whilst the ABS is focussed on the ongoing improvement of our own internal 
quality assurance processes and looking for international collaborations where 
possible, there is also an imperative to look to externalise best practice and 
guidelines to the Australian community. The ABS Data Quality Framework is one 
of these 'best practice’ guides that has been formally adopted by other Australian 
government departments for use as one part of an overall quality management 
strategy. 


The Council of Australian Governments mandated in 2008 that the ABS Data 
Quality Framework was to be used to assess the performance indicators of the 
National Reporting Framework "National Agreements" which influence funding 


to State and Territories in Australia*. The ABS Data Quality Framework would be 
used to declare the quality of the data provided for the performance indicators in 
the form of accompanying Quality Statements that assess the quality of the data 
in regards to the performance indicator. This announcement was a major 
achievement in the externalisation of the ABS Data Quality Framework. 


The quality statements that are required for COAG reporting are similar to 
the ABS' own quality declarations with the major difference being that quality 
statements for COAG are specific quality assessments on how well the data 
meets the performance indicator rather than the ABS' own quality declarations 
which are very general in the information presented due to the specific purpose 
of the data by a user being unknown. 


COAG mandating the use of the ABS Data Quality Framework has led to an 
increase in the use of the ABS Data Quality Framework by government 
departments for the supply of data reports for the performance indicators. 
However a further consequence has been an increase in interest from 
government departments in relation to incorporating the Framework into their 
internal processes. 


The announcement by COAG that the ABS Data Quality Framework would 
be used to declare the quality of the data provided for the performance 
indicators led to a flurry of questions about the Framework. At the time of the 
announcement the ABS Data Quality Framework, whilst mentioned on the ABS 
website was not easily accessible. The ABS rectified this issue by producing the 
ABS Data Quality Framework, May 2009 (cat. no. 1520.0) product which 
provides an overview of all the dimensions and how to use the Framework. This 
was the start of the ABS' commitment to aiding COAG in its quest to have a more 
transparent National Reporting Framework. 


The ABS has had numerous roles in the development of quality statements 
for COAG. The roles have included that of data provider, advisor and assessor. 
The roles have evolved over the past twelve months with everyone involved in 
the COAG reporting process learning progressively. 


In the role of data provider the ABS, like other data providers for COAG, is 
required to provide quality statements to accompany the indicator data. These 
statements were written by the relevant areas in the ABS. They covered all the 
seven dimensions and commented on the fitness for purpose of the data for the 
indicator specified. As with other providers of data and quality statements, the 
first year of reporting was quite new and over time it is expected that the content 


? The Council of Australian Governments (COAG) is the peak intergovernmental 
forum in Australia. COAG comprises the Prime Minister, State Premiers, 
Territory Chief Ministers and the President of the Australian Local Government 
Association (ALGA). The role of COAG is to initiate, develop and monitor the 
implementation of policy reforms that are of national significance and which 
require cooperative action by Australian governments. 


of quality statements will be refined and become more explicitly set by COAG as 
to requirements. 


In the role of advisor the ABS has acted as both a teacher and consultant. 
The ABS has provided seminars to government departments on the ABS Data 
Quality Framework and how it can be used for their own internal purposes and 
for COAG reporting purposes. These seminars have been interactive and have 
led to departments requesting workshops and training courses for more tailored 
help. 


Consultancy work differs from teaching in that it's an on-going specific role. 
The ABS has acted as a consultant to two of the groups that work on the reports 
that are presented to COAG. Advice was provided on possible content expected 
within each dimension of the ABS Data Quality Framework. This involved 
reviewing draft quality statements and considering other information not 
mentioned that might have a bearing on whether the data was fit for COAG's 
purpose. This advice was considered and where appropriate the questions for 
more information were directed to the suppliers of the data. The ABS also 
supplied a staff member to one of these groups in an outposting arrangement to 
explicitly help in the set up of this new reporting program. This officer has acted 
as an advisor and also facilitated communication between the group and the ABS 
where necessary. 


The expertise of the ABS was requested by COAG in the role of assessor. 
The ABS review quality statements produced (including their own) and provide 
feedback on how well the data meets the requirements along with suggested 
improvements for quality statements for subsequent rounds of reporting. The 
role has also seen the ABS helping to identify data gaps through analysis of the 
quality statements and the fitness for purpose of the data for the performance 
indicators, such as the ABS increasing sample size in some of our population 
surveys to meet COAG requirements of State and Territory level data for 
comparison. It has also led to new collections being developed primarily for 
COAG reporting purposes. These changes have been very quick to be 
implemented to the survey program of the organisation. 


3. New Initiative to Aide the Creation of Quality Statements 


The first cycle of the National Reporting Framework National Agreements 
and their subsequent quality statements was completed in 2009. Feedback from 
users of the ABS Data Quality Framework for COAG reporting purposes, indicate 
they have found it difficult to know what to report beneath each dimension and 
the usefulness of Institutional Environment. As a result the ABS developed a 
quality tool, Data Quality Online, which will help users draft quality statements 
using the ABS Data Quality Framework by providing contextual information 
within each dimension to guide the content. 


The contextual information beneath each dimension is in the form of 
questions. These questions are for consideration when constructing a quality 


statement to declare the quality of the data in the context of the performance 
indicator (or data need). They may require the data provider to refer back to 
existing internal performance indicators of process quality and report on these 
within the quality statement. The questions may also identify information that 
hadn't previously been considered for reporting purposes on the quality of the 
data. They may also identify gaps in the knowledge of the quality of the process 
and hence identify a need for the development of additional performance 
indicators by the user on their internal processes. 


Aside from the formal reporting of quality statements for the performance 
indicators for COAG reporting, government departments have shown an interest 
in the use of the ABS Data Quality Framework to quality assess their own data 
holdings. This has been especially in regards to the quality assessment of their 
administrative data. In response to this need the Data Quality Online tool will 
also contain information on the use of the ABS Data Quality Framework for more 
generic purposes other than specific COAG reporting. 


The contextual information contained within the tool (Data Quality Online) 
for formal reporting purposes was developed in conjunction with feedback from 
the first round of COAG reporting. The feedback came from those users who had 
to create quality statements for reporting to COAG and those agencies 
responsible for collating the quality statements. The generic information for 
general use of the ABS Data Quality Framework has been developed but has yet 
to undergo user testing. This testing is scheduled to be completed by June 2010. 


The first release of Data Quality Online with information pertaining to the 
COAG reporting quality statements occurred on the 25th of March 2010, on the 
national statistical service website <www.nss.gov.au>. This occurred after 
testing of the system and the questions were undertaken by both the ABS and 
some external agencies involved in the COAG reporting. 


Further development on the Data Quality Online tool in respect to both 
COAG reporting and general use of the ABS Data Quality Framework for 
declaring quality will occur throughout 2010 once further feedback from users 
has been obtained. 


4. ABS Data Quality Framework and the Development of Performance 
Indicators 


The performance indicators for COAG reporting are to be reviewed each 
year to assess whether they are delivering the results required in terms of the 
outcomes they are supposed to reflect. The assessment of the performance 
indicator will come directly from the quality statements that use the ABS Data 
Quality Framework for assessing. In this case data gaps for those indicators that 
are not able to be delivered upon due to current data constraints may lead to 
underlying data sources being expanded to be able to supply the required 
information. An example of this is an increase in funding to the ABS by COAG to 
increase the sample size of at least one of our population surveys. 


Performance indicators will also be reviewed in regards to whether they 
meet the outcomes required. It may be that the original performance indicator 
specified by COAG does not make sense in the context of real world information 
or what is now known from experience of the first round of reporting. 


Where data gaps are identified in the reporting for outcomes, new 
performance indicators will be developed for reporting upon. Alternatively where 
existing performance indicators need to be removed because they are no longer 
fit for purpose this will also occur. 


Data Quality Online has been developed quickly to meet the specific COAG 
reporting need for information on how to use the ABS Data Quality Framework. 
As a result, further feedback from the 2009 round of quality statements and 
subsequent rounds will provide opportunities for further refinement and 
improvements to the content within each dimension. These improvements may 
highlight additional performance indicators, both at the National Agreement and 
process orientated levels, required for the ongoing quality reporting to COAG. 


For government departments using the ABS Data Quality Framework to 
quality assure their own data holdings, the questions within each dimension will 
promote continuous improvement in the underlying processes by highlighting 
gaps in the performance indicators of these processes. For example, questions 
relating to adjustments to data and impacts of these adjustments on the data will 
prompt further investigation by data holders if this information is not already 
available. Another example would be questions relating to the percentages of 
missing data, be it through non-response or incorrect coding etc., may draw 
attention to missing performance indicator information of the process. 


As a result of the use of the ABS Data Quality Framework by government 
departments for reporting to COAG, or for general quality assessment of data 
holdings, users will become more familiar and understand their data better. This 
is through both reporting on existing known quality aspects of the data in 
question and also the development of additional performance indicators for 
either internal processes or for the National Agreements to help with declaring 
the quality of the data. 


5. Future opportunities for externalisation of the ABS Data Quality 
Framework 


The up take of the ABS Data Quality Framework in the government sector 
for assessment purposes and hence performance indicator development for both 
the formal creation of quality statements for COAG reporting and for internal 
quality assurance processes has been a great achievement of the ABS'. The ABS 
is looking for further opportunities to expand the use of the ABS Data Quality 
Framework within the Australian community. The latest endeavour includes 
opportunities to provide information to the private sector on the ABS Data 
Quality Framework and its uses through participation in various forums. In 
particular, the ABS has submitted a synopsis on the ABS Data Quality 
Framework to the Australian Market and Social Research Society which may be 
distributed to its members for consideration as a discussion topic at various 
forums in the future. 


It is hoped that through promotion of the ABS Data Quality Framework to 
the Australian Market and Social Research Society that the ABS Data Quality 
Framework may be used throughout the private sector to also declare and assess 
their data collections. Providing information to the Society on the use of the ABS 
Data Quality Framework by government departments may help the private sector 
when supplying data to these government departments by having a consistent 
framework on which to declare the quality of the data being supplied. 


The direction for the Data Quality Online tool has yet to be fully explored 
and further discussions with external client agencies, other than COAG related 
ones, could result in the tool being expanded to cater for other requirements. 


These possible future developments may help cement the ABS Data Quality 
Framework as the framework that is used for quality assurance throughout 
Australian Government departments, not only for COAG purposes, but for 
general internal practices and processes, as well as the private sector in 
Australia. 


6. Conclusion 


The adoption of the ABS Data Quality Framework by the Council of 
Australian Governments for use in reporting the quality of the performance 
indicators for their National Agreements has provided the ABS with an 
opportunity to expand their statistical leadership role. The development of the 
Data Quality Online has been in response to the needs of users in other 
government organisations who have been requesting greater clarification on 
using the ABS Data Quality Framework. This is for both meeting COAG 
requirements as well as for their own internal processes. The Data Quality 
Online tool helps users draft quality statements for assessment purposes by 
having detailed contextual questions beneath each of the dimensions of the ABS 


Data Quality Framework. Thus, the ABS is enabling the government departments 
to develop performance indicators at both a national level (National Agreements) 
and at the individual process levels. This is in line with the ABS' mission of 
assisting and encouraging informed decision making, research and discussion 
within governments and the community, by leading a high quality, objective and 
response national statistical service 
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